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Abstract. We study the generating function for the number of permutations on 
n letters containing exactly r ^ occurrences of 132. It is shown that finding this 
function for a given r amounts to a routine check of all permutations in S^r- 

2000 Mathematics Subject Classification: Primary 05A05, 05A15; Secondary 
05C90 



1. Introduction 

Let 7r € S n and t € S m be two permutations. An occurrence of r in 7r is a 
subsequence 1 ^ %\ < i% < • • • < i m ^ n such that . . . , 7r(i m )) is order- 

isomorphic to t; in such a context, t is usually called a pattern. 

Recently, much attention has been paid to the problem of counting the number 
■0r ( n ) °f permutations of length n containing a given number r of occurrences of 
a certain pattern r. Most of the authors consider only the case r = 0, thus studying 
permutations avoiding a given pattern. Only a few papers consider the case r > 0, 
usually restricting themselves to the patterns of length 3. In fact, simple algebraic 
considerations show that there are only two essentially different cases for t e S3, 
namely, r = 123 and r = 132. Noonan [No] has proved that ^ 23 (n) = f ( n 2 " 3 )- 
A general approach to the problem was suggested by Noonan and Zeilberger [NZ] ; 
they gave another proof of Noonan's result, and conjectured that 

123/ * 59n 2 + 117n+ 100 / 2n 



2n(2n- l)(n + 5) \n - 4 



and 'ijjl 32 (n) = ( ^3)- The latter conjecture was proved by Bona in [B2]. A general 
conjecture of Noonan and Zeilberger states that ip^ ( n ) i s -P- recursive in n for any 
r and r. It was proved by Bona [Bl] for r = 132. However, as stated in [Bl], a 
challenging question is to describe ip^(n), t € S3, explicitly for any given r. 

In this note we suggest a new approach to this problem in the case r = 132, 
which allows to get an explicit expression for ^rin) = i/>^ 32 (n) for any given r. 
More precisely, we present an algorithm that computes the generating function 
^ r (x) = X)n>o ''Pri'n^x 71 for any r 0. To get the result for a given r, the algorithm 
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performs certain routine checks for each element of the symmetric group S^r- The 
algorithm has been implemented in C, and yielded explicit results for 1 < r < 6. 

The authors are sincerely grateful to M. Fulmck and A. Robertson for inspiring 
discussions. 

2. Preliminary results 

To any it £ S n we assign a bipartite graph G x in the following way. The vertices 
in one part of G^, denoted V\, are the entries of 7r, and the vertices of the second 
part, denoted V3, are the occurrences of 132 in tt. Entry i £ V\ is connected by 
an edge to occurrence j £ V 3 if i enters j. For example, let tt — 57614283, then tt 
contains 5 occurrences of 132, and the graph G T is presented on Figure 1. 

576 142 143 183 283 

m 

5 7 6 1 4 2 8 3 

Fig. 1. Graph G n for tt = 57614283 

Let G be an arbitrary connected component of G^, and let V be its vertex set. 
We denote V x = V n V u V 3 = V n V 3 , h = |VL|, t 3 = \V 3 \. 

Lemma 1. For any connected component G of G^ one has t\ < 2t 3 + 1. 

Proof. Assume to the contrary that the above statement is not true. Consider the 
smallest n for which there exists ir £ S n such that for some connected component 
G of G^ one has 

h > 2t 3 + 1. (*) 

Evidently, G contains more than one vertex, since otherwise ti = 1, t 3 = 0, which 
contradicts (*). Let I be the number of leaves in G (recall that a leaf is a vertex of 
degree 1). Clearly, all the leaves belong to V\\ the degree of any other vertex in V\ 
is at least 2, while the degree of any vertex in V3 equals 3. Calculating the number 
of edges in G by two different ways, we get I + 2(t\ — I) < 3^3, which together with 
(*) gives I > t 3 + 2. This means that there exist two leaves u,v £ V\ incident to 
the same vertex a £ V3. 

Let w £ V\ be the third vertex incident to a. If w is a leaf, then G contains 
only four vertices a, u, v, w, and hence t\ = 3, t 3 = 1, which contradicts (*). Hence, 
the degree of w is at least 2. Delete the entries u, v from n and consider the 
corresponding permutation ir' £ 5„_2- Denote by G' the connected component of 
Gtt' containing w. Since the degree of w in G was at least 2, we see that G' is 
obtained from G by deleting vertices u, v, and a. Therefore, t[ =ti — 2,t' 3 =t 3 — 1, 
and hence t[ > 2t' 3 + 1, a contradiction to the minimality of n. □ 

Denote by G™ the connected component of G^ containing entry n. Let . . . , 

n(i s ) be the entries of ir belonging to G™, and let a = ov £ S s be the corresponding 
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permutation. We say that ■ ■ ■ ,7r(i s ) is the kernel of 7r and denote it ker7r; 

a is called the shape of the kernel, or the kernel shape, s is called the size of the 
kernel, and the number of occurrences of 132 in ker7r is called the capacity of the 
kernel. For example, for tt = 57614283 as above, the kernel equals 14283, its shape 
is 14253, the size equals 5, and the capacity equals 4. 

The following statement is implied immediately by Lemma 1. 

Theorem 1. Let tt £ S n contain exactly r occurrences of 132, then the size of the 
kernel of tt is at most 2r + 1 . 

We say that p is a kernel permutation if it is the kernel shape for some permu- 
tation tt. Evidently p is a kernel permutation if and only if a p = p. 

Let p £ S s be an arbitrary kernel permutation. We denote by S(p) the set of all 
the permutations of all possible sizes whose kernel shape equals p. For any tt £ S(p) 
we define the kernel cell decomposition as follows. The number of cells in the 
decomposition equals s(s + 1). Let ker7r = n(ii), . . . , 7r(i s ); the cell C m i — C m ;(7r) 
for 1 < I < s + 1 and 1 ^ m < s is defined by 

C m ;(?r) = {ir(j): < j < i u Tr{i p -i( m -i)) < ^U) < t(V-'(™))}' 

where i = 0, i s+ \ = n+ 1, and a(0) = for any a. If n coincides with p itself, then 
all the cells in the decomposition are empty. An arbitrary permutation in S(p) is 
obtained by filling in some of the cells in the cell decomposition. A cell C is called 
infeasible if the existence of an entry a £ C would imply an occurrence of 132 that 
contains a and two other entries x,y £ ker-zr. Clearly, all infeasible cells are empty 
for any tt £ S(p). All the remaining cells are called feasible; a feasible cell may, 
or may not, be empty Consider the permutation tt = 67382451. The kernel of 7r 
equals 3845, its shape is 1423. The cell decomposition of n contains four feasible 
cells: Ci 3 = {2}, C u = 0, C 15 = {1}, and C 41 = {6, 7}, see Figure 2. All the other 
cells are infeasible; for example, C32 is infeasible, since if a £ C32, then an'(i2)7T'(i4) 
is an occurrence of 132 for any ir' whose kernel is of shape 1423. 
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Fig. 2. Kernel cell decomposition for tt £ 5(1423) 
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As another example, permutation 7r = 101171246583921 belongs to the same 
class 5(1423). Its kernel is 7 12 8 9, and the feasible cells are C13 = {4, 6, 5}, C14 = 
{3}, Ci B = {2,l}, C 4 i = {10,ll}. 

Given a cell dj in the kernel cell decomposition, all the kernel entries can be 
positioned with respect to dj- We say that x = ir(ik) £ kcr7r lies below dj if 
p(k) < i, and above Cij if p(k) ^ i. Similarly, x lies to the left of Cij if k < j, and 
to the right of dj if k j. As usual, we say that x lies to the southwest of Cjj if it 
lies below dj and to the left of it; the other three directions, northwest, southeast, 
and northeast, are defined similarly. 

The following statement plays a crucial role in our considerations. 

Lemma 2. Let n £ S(p) and ir(ik) £ ker7r, then any cell C m i such that I > k and 
m > p(k) is infeasible. 

Proof. Assume to the contrary that there exist I and m as above such that C m ; 
is feasible, and consider the partition of the entries in kcr7r into two subsets: yt\ 
containing the entries of kcr tt that lie to the southwest of C m i and X2 containing 
the rest of the entries. Observe that ir{ik) 7^ n, since n ^ m > p{k) = n(ik)- 
Moreover, n(ik) £ x\ and n £ X2- Since n(ik) and n belong to the same connected 
component of G n , there exists at least one occurrence of 132 whose elements are 
distributed between >c\ and x^.. Let a denote the minimal entry in this occurrence, 
let c denote its maximal entry, and let b denote the remaining entry. 
Evidently, a € x± . Assume first that c <E X2 ■ 

If c lies to the left of C m i , then the existence of z € C m i would imply that acz is 
an occurrence of 132, and hence C m i is infeasible. 

If c lies to the northeast of C m i and b lies above C m i, then the existence of 
z € C m i would imply that zeb is an occurrence of 132, and hence C m i is infeasible. 
If b lies below C m i, then the existence of z € C m i would imply that azb is an 
occurrence of 132, and hence C m ; is infeasible. 

If c lies to the southeast of C m i , then the existence of z € C m i would imply that 
azc is an occurrence of 132, and hence C m i is infeasible. 

It remains to consider the case c £ k\, which means that b belongs to ^ 2 and 
lies to the southeast of C m /. Hence, the existence of z £ C m i would imply that azb 
is an occurrence of 132, and hence C m i is infeasible. □ 

As an easy corollary of Lemma 2, we get the following proposition. Let us define 
a partial order -< on the set of all feasible cells by saying that C m i -< C m n' 7^ C m i 
if m ^ ml and / ^ V . 

Lemma 3. -< is a linear order. 

Proof. Assume to the contrary that there exist two feasible cells C m i and dn'i' such 
that I < V and m < m', and consider the entry x = ir(ii) £ ker7r. By Lemma 2, 
x > 7r(i p -i( m /_i)), that is, x lies above the cell C m >i>, since otherwise C m >v would 
be infeasible. For the same reason, y = TT(i p -i( m >-u) lies to the right of C m ni, and 
hence to the right of x. Therefore, the existence of z £ C m i would imply that zxy 
is an occurrence of 132, and hence C m i is infeasible, a contradiction. □ 

Consider now the dependence between two nonempty feasible cells lying on the 
same horizontal or vertical level. 
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Lemma 4. Let C m i and C m i> be two nonempty feasible cells such that I < I'. Then 
for any pair of entries a G C m i, b G C m v , one has a > b. 

Proof. Assume to the contrary that there exists a pair a € C m i, b G C m i' such that 
a < b. Consider the entry x = 7r(i;) G kcr7r. By Lemma 2, x > b, since otherwise 
C m v would be infcasiblc. Hence axb is an occurrence of 132, which means that 
both a and b belong to ker7r, a contradiction. □ 

Lemma 5. Let C m i and C m >i be two nonempty feasible cells such that m < m' . 
Then any entry a G C m i lies to the right of any entry b G C m n. 

Proof. Assume to the contrary that there exists a pair a G C m i, b G C m n such 
that a lies to the left of b. Consider the entry y = ir(i p -i( m >-i)) G ker7r. By 
Lemma 2, y lies to the right of b, since otherwise C m >i would be infeasible. Hence 
aby is an occurrence of 132, which means that both a and b belong to ker-zr, a 
contradiction. □ 

Lemmas 3-5 yield immediately the following two results. 

Theorem 2. Let G be a connected component of distinct from 67™. Then all 
the vertices in V\ belong to the same feasible cell in the kernel cell decomposition of 
ir. 

Let F(p) be the set of all feasible cells in the kernel cell decomposition corre- 
sponding to permutations in S(p), and let f(p) = \F(p)\. We denote the cells in 
F(p) by C 1 , . . . , C^ pS) in such a way that C % < C j whenever i < j. 

Theorem 3. For any given sequence a\, . . . , a/( p ) of arbitrary permutations there 
exists 7r G S{p) such that the content of C l is order-isomorphic to m. 



3. Main Theorem and explicit results 

Let p be a kernel permutation, and let s(p), c(p), and f{p) be the size of p, the 
capacity of p, and the number of feasible cells in the cell decomposition associated 
with p, respectively. Denote by K the set of all kernel permutations, and by K t the 
set of all kernel shapes for permutations in St ■ The main result of this note can be 
formulated as follows. 

Theorem 4. For any r > 1, 



/ _ Hp) 

* r (x)= ]T * s(rf ]T n*M x ))- (**) 

peK 2r+ i \ riH \-ri( p )=r-c(p) j=l 



where rj ^ for 1 ^ j ^ f(p)- 

Proof. For any p G K, denote by ^ p {x) the generating function for the number of 
permutations in 7r G S n n S(p) containing exactly r occurrences of 132. Evidently, 
^ r (x) = "Ylip^x ^r( x )- To hnd ^p(x), recall that the kernel of any ir as above 
contains exactly c(p) occurrences of 132. The remaining r — c(p) occurrences of 132 
are distributed between the feasible cells of the kernel cell decomposition of ir. By 
Theorem 2, each occurrence of 132 belongs entirely to one feasible cell. Besides, it 
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follows from Theorem 3, that occurrences of 132 in different cells do not influence 
one another. Therefore, 

Hp) 

riH hr /(p) =r-c(p) j=l 

and we get the expression similar to (**) with the only difference that the outer 
sum is over all p e K. However, if p e K t for t > 2r + 1, then by Theorem 1, 
c(p) > r, and hence *£(a;) = 0. □ 

Theorem 4 provides a finite algorithm for finding ^ r (x) for any given r > 0, 
since we have to consider all permutations in S^r+i, and to perform certain routine 
operations with all shapes found so far. Moreover, the amount of search can be 
decreased substantially due to the following proposition. 

Proposition. The only kernel permutation of capacity r ^ 1 and size 2r + 1 is 
2r-12r + 12r-32r . . . 2r-2j-3 2r-2j ... 142. its contribution to * r (x) eguafo 

This proposition is proved easily by induction, similarly to Lemma 1. The feasible 
cells in the corresponding cell decomposition are C27— 2j+i,2j+i) J ' = 0, . . . , r, and 
Ci.2r+2, hence the contribution to x f' r (x) is as described. 

By the above proposition, it suffices to search only permutations in S 2r - Below 
we present several explicit calculations. 

Let us start from the case r = 0. Observe that (**) remains valid for r = 0, 
provided the left hand side is replaced by *& r (%) — 1; subtracting 1 here accounts for 
the empty permutation. So, we begin with finding kernel shapes for all permutations 
in Si. The only shape obtained is pi = 1, and it is easy to see that s(pi) = 1, 
c(pi) = 0, and /(pi) = 2 (since both cells Cn and C12 are feasible). Therefore, we 
get ^o(x) — 1 = x^l(x), which means that 



= 2x ' 

the generating function of Catalan numbers. 

Let now r = 1. Since permutations in S2 do not exhibit kernel shapes distinct 
from pi, the only possible new shape is the exceptional one, p 2 = 132, whose 
contribution equals rr 3 ^^). Therefore, (**) amounts to 

*i(a;) = 2x$ (x)$i(x) +x 3 *g(x), 

and we get the following result. 

Corollary 1 (Bona [B2, Theorem 5]). 

*i(a;) = \ (x - 1 + (1 - 3a;)(l - 4.x)- 1 / 2 ) ; 

equivalently, 

(2n-i 
for n ^ 3. 

Let r — 2. We have to check the kernel shapes of permutations in S4. Exhaustive 
search adds four new shapes to the previous list; these are 1243, 1342, 1423, and 
2143; besides, there is the exceptional 35142 e S5. Calculation of the parameters 
s, c, / is straightforward, and we get 
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Corollary 2. 

*2(ar) = ^ (a 2 + 3a - 2 + (2a 4 - 4a 3 + 29a; 2 - 15a; + 2)(1 - Ax) ~ 3/2 ) ; 
equivalently, 

, , , n 3 + 17n 2 - 80n + 80 /2n - 6\ 

^ 2(n) = 2^1) U-aJ 

/or n 5? 4. 

Let r = 3,4,5,6; exhaustive search in 5*6, S%, S\q, and reveals 20, 104, 503, 
and 2576 new nonexceptional kernel shapes, respectively, and we get 

Corollary 3. Let 3 ^ r ^ 6, then 

*r(*) = ^ (Pr(x) + Q r (x)(l - 4x)- r+1 ^) , 



where 



P 3 (x) = 2a 3 -5a 2 + 7a-2, 

P 4 (a) = 5a 4 - 7a 3 + 2a 2 + 8a - 3, 

P 5 (x) = 14a 5 - 17a 4 + a 3 - 16a 2 + 14a - 2 



and 



Q 3 (a) = -22a 6 - 106a 5 + 292a 4 - 302a 3 + 135a 2 - 27a + 2, 
Q 4 (a) = 2a 9 + 218a 8 + 1074a 7 - 1754a 6 + 388a 5 + 1087a 4 

- 945a 3 + 320a 2 - 50a + 3, 
Q 5 (x) = -50a 11 - 2568a 10 - 10826a 9 + 16252a 8 - 12466a 7 + 16184a 6 - 16480a 5 

+ 9191a 4 - 2893a 3 + 520a 2 - 50a + 2. 

Equivalently, 

i < \ n t \ (2"-3r)! 
ip r [n) = R r (n) 



n\r\{n — r — 2)\ 1 
for n ^ r + 2, where 

R 3 (n) = n 6 + 51n 5 - 407n 4 - 99n 3 + 7750n 2 - 22416n + 20160, 
Ri{n) = n 9 + 102n 8 - 282n 7 - 12264n 6 + 32589n 5 + 891978n 4 

- 7589428n 3 + 25452024n 2 - 39821760n + 23950080, 
R 5 (n) = n 12 + 170n n + 1861n 10 - 88090n 9 - 307617n 8 + 27882510n 7 

- 348117457n 6 + 2119611370n 5 - 6970280884n 4 

+ 10530947320n 3 + 2614396896n 2 - 30327454080n + 29059430400. 



The expressions for P&(x), Qe(x), and Rs(n) are too long to be presented here. 
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4. Further results and open questions 



As an easy consequence of Theorem 4 we get the following results due to Bona 



Corollary 4. Let r ^ 0, then ^ r {x) is a rational function in the variables x and 



where P r (x) and Q r (x) are polynomials and 1 — Ax does not divide Q r {x). We 
were unable to prove this result; however, it stems almost immediately from the 
following conjecture. 

Conjecture 1. For any kernel permutation p ^ 1, 



Indeed, it is easy to see that ^ r (x) enters the right hand side of (**) with the 
coefficient 2x^ {x), which is a partial contribution of the kernel shape p\ = 1. Since 
1 — 2x^q(x) = VI — 4a; , we get by induction from (**) that VI — Ax^ r (x) equals 
the sum of fractions whose denominators are of the form x d (l — 4x) r ~ c ( p )~-^ p )/ 2 , 
where d ^ f(p) . On the other hand, each fraction is multiplied by x s ( p \ hence if 
s{p) ^ f(p) as conjectured, then x d in the denominator is cancelled. The maximal 
degree of (1 — Ax) is attained for p — pi, and is equal to r — 1, and we thus arrive 
at (* * *). 

In view of our explicit results, we have even a stronger conjecture. 

Conjecture 2. The polynomials P r (x) and Q r {x) in (* * *) have halfinteger coef- 
ficients. 

Another direction would be to match the approach of this note with the previous 
results on restricted 132-avoiding permutations. Let <fr r (x;k) be the generating 
function for the number of permutations in S n containing r occurrences of 132 and 
avoiding the pattern 12 ... k G Sk- It was shown previously that <fr r (x; k) can be 
expressed via Chebyshev polynomials of the second kind for r = ([CW]) and 
r = 1 ([MV]). Our new approach allows to get a recursion for <& r (x; k) for any given 
r ^ 0. 

Let p be a kernel permutation, and assume that the feasible cells of the kernel cell 
decomposition associated with p are ordered linearly according to -<!. We denote by 
lj (p) the length of the longest increasing subsequence of p that lies to the north-east 
from CK For example, let p = 1423, as on Figure 2. Then l\{p) = 1, h{p) = 2, 



[Bl 



In fact, Bona has proved a stronger result, claiming that 



* r (a;) - P r (x) + Q r {x)(l - Ax)- r+1 / 2 , 




s{p) > f{p). 



h{p) - 1, h(p) = 0. 



Theorem 5. For any r ^ 1 and k ^ 3 




f(p) 
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where rj > for 1 ^ j < f(p) and $ r (x; to) = /or to < 0. 

As in the case of ^ r (x), the statement of the theorem remains valid for r = 0, 
provided the left hand side is replaced by $ r (x; k) — 1. This allows to recover known 
explicit expressions for <I> r (x; fc) for r = 0, 1, and to get an expression for r = 2, 
which is too long to be presented here. 

This approach can be extended even further, to cover also permutations con- 
taining r occurrences of 132 and avoiding other permutations in Sk, for example, 
23... fcl. 

References 

[Bl] M. Bona, The number of permutations with exactly r 132-subsequences is P-recursive in 

the size!, Adv. Appl. Math. 18 (1997), 510-522. 
[B2] M. Bona, Permutations with one or two 132-subsequences, Discr. Math 181 (1998), 267— 

274. 

[CW] T. Chow and J. West, Forbidden subsequences and Chebyshev polynomials, Discr. Math. 
204 (1999), 119-128. 

[MV] T. Mansour and A. Vainshtein, Restricted permutations, continued fractions, and Cheby- 
shev polynomials, Electron. J. Combin. 7 (2000), #R17. 

[No] J. Noonan, The number of permutations containing exactly one increasing subsequence of 
length three, Discr. Math. 152 (1996), 307-313. 

[NZ] J. Noonan and D. Zeilberger, The enumeration of permutations with a prescribed number 
of "forbidden" patterns, Adv. Appl. Math. 17 (1996), 381-407. 



